Structure-preserving neural networks have become widely popular in many areas of science due to their ability to incorporate physical and geometric biases into the learning process and/or neural architecture. Doing so allows one to create models with less parameters that accurately learns dynamics and generalises to unseen data. In this presentation we will explore how one can directly incorporate some geometric biases such as volumepreservation, contractivity, symplectic and conformal symplectic structure into neural networks for learning maps from data with known geometry.